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ABSTRACT 

Abundant  experimentation  has  been  conducted  to  compare  decision-making  performance  of 
Edge  versus  Hierarchical  organizations  within  the  C2  research  community.  The  most 
commonly  utilized  testing  tool  has  been  the  ELICIT  platform,  which  enables  provision  and 
sharing  of  data  that  is  relevant  to  solve  a  problem.  In  order  to  enrich  the  interaction  space 
provided  by  ELICIT,  we  developed  a  complementary  tool  we  term  CIP  (Common 
Identification  Picture),  which  provides  a  setting  for  problem  solution  sharing  (attack 
identification).  We  use  the  combination  of  ELICIT  and  CIP  to  examine  interactions  at  both 
the  information  domain  (ELICIT)  and  the  cognitive  domain  (ELICIT  and  CIP).  We  then 
make  comparisons  between  Edge  and  Hierarchical  organizations  performance. 

Our  findings  suggest  that  while  Edge  organizations’  problem-solving  abilities  outperform 
those  of  Hierarchical  organizations  if  knowledge  sharing  is  facilitated,  the  overall 
performance  of  both  kinds  of  organizations  is  considerably  improved  when  the 
complementary  tool  is  implemented. 

One  possible  explanation  for  our  results  is  that  Edge  organizations  do  better  than 
Hierarchies  because  the  enrichment  of  the  interaction  space  is  better  exploited  by  the  kind 
of  organization  featuring  a  more  complete  pattern  of  interactions  and  freedom  to 
collaborate,  as  opposed  to  the  more  restricted  one. 

Regarding  the  use  of  CIP,  we  argue  that  enhanced  communication  channels  facilitate 
collaboration;  increase  quality  of  interactions;  enhance  distribution  of  information  and, 
therefore,  improve  awareness  and  understanding  (Alberts  and  Hayes,  2006). 
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INTRODUCTION 

The  advent  of  new  information  technologies  has  enabled  transformational  changes  on  the 
way  war  is  conducted  today.  In  fact,  US  DoD’s  transformation  is  about  continuous 
adaptation  to  the  Information  Age  (Alberts  and  Hayes,  2003)  in  order  to  capitalize  on  the 
new  resources  at  hand.  Novel  approaches  point  out  that  by  leveraging  the  power  of 
enhanced  information  availability,  modem  militaries  are  able  to  hasten  their  decision  cycles 
to  the  point  that  it  becomes  extremely  difficult  for  adversaries  to  react  in  a  timely  manner. 
In  practice,  however,  this  capability  has  persisted  elusive  as  it  involves  a  twofold  challenge. 
On  the  one  hand,  it  requires  being  able  to  deal  with  the  coevolution  of  mission  capability 
packages  comprising  concepts  of  operation,  approaches  to  command  and  control, 
organizational  forms,  doctrine,  corresponding  C4ISR,  weapons,  and  logistic  systems 
(Alberts  et  al,  2001).  While  on  the  other,  it  demands  an  advanced  understanding  of 
information  superiority  and  network  centric  warfare  concepts,  which  often  remain  as 
abstract  notions  and  are  found  difficult  to  apply  to  military  operations  and  organizations. 

It  seems  that  superior  unit  performance  is  a  key  element  to  persuade  leaders  to  become 
more  aggressive  in  adopting  these  new  Information  Age  military  concepts.  Therefore, 
research  efforts  on  the  usage  of  IT  tools  under  different  organizational  configurations  and 
their  effects  on  team  problem-solving  capabilities  has  become  a  topic  of  high  significance 
for  the  military.  To  deal  with  this  need,  a  scientific  community  led  by  the  Command  and 
Control  Research  Program  (CCRP)  of  the  US  DoD  is  exploring  performance  differences, 
within  a  networked  environment,  between  a  flat  organizational  structure  where  every 
member  has  the  ability  to  access  all  the  available  information,  herein  termed  Edge 
Organization,  versus  a  traditional  Hierarchical  organization,  where  information  flows  are 
constrained  by  the  procedural  restrictions  of  the  chain  of  command  (Alberts  and  Hayes, 
2003).  As  part  of  this  effort,  CCRP  has  sponsored  the  development  of  ELICIT1,  a  software 
application  designed  for  conducting  human-in-the-loop  experiments  focused  on  information 
and  social  domain  phenomena  (Martin  and  Me  Ever,  2008). 


1  ELICIT  stands  for  “Experimental  Laboratory  for  Investigating  Collaboration,  Information-sharing  and 
Trust”. 
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Within  this  scope,  this  paper  aims  to  present  experimental  research  conducted  by  the 
Military  Polytechnic  Academy  of  the  Chilean  Army,  which  used  the  ELICIT  platform  to 
contrast  the  performance  of  Edge  vis  a  vis  Hierarchical  organizations  in  both  the 
information  and  the  cognitive  domain.  For  the  latter,  we  combine  ELICIT,  — whereby 
participants  can  share  information  about  potential  threats —  with  a  complementary  tool  we 
call  Common  Identification  Picture  (CIP),  which  enables  participants  to  share  their 
knowledge  of  the  situation  by  posting  their  threat  identifications. 

WHY  EDGE  VERSUS  HIERARCHICAL  ORGANIZATIONS? 

The  NATO  NEC  Maturity  Model  (NATO,  2010)  has  associated  the  ability  to  adopt  diverse 
C2  approaches  -conflicted,  de-conflicted,  coordinated,  collaborative  and  edge-  with  five 
different  levels  of  C2  maturity  -levels  1  thru  5.  Superior  maturity  levels  correspond  to  the 
ability  of  an  entity  to  adopt  a  broader  variety  of  C2  approaches  and  to  select  the  appropriate 
one  depending  on  the  ongoing  situation.  These  approaches  differ  from  one  another  in  one 
or  more  of  three  interconnected  dimensions  that  make  up  the  C2  approach  space,  namely: 
allocation  of  decision  rights;  patterns  of  interaction;  and  distribution  of  information 
(Alberts  and  Hayes,  2006).  The  extremes  of  the  C2  approach  space  are  associated  with 
differences  in  centralization  of  command  and  control  practices.  Thus,  the  most  centralized 
mode  (C2  Maturity  Level  1)  can  only  apply  the  Conflicted  C2  approach;  whereas  the  less 
centralized  one  (C2  Maturity  Level  5)  is  able  to  apply  either  De-Conflicted,  Coordinated, 
Collaborative  or  Edge  C2  approach  (NATO  SAS-065,  2010).  Since  this  model  associates 
the  highest  maturity  level  (level  5)  with  Edge  organizations  and  the  lowest  maturity  with 
Hierarchies,  it  is  expected  that,  within  a  networked  environment,  Edge  organizations 
perform  better  than  the  traditional  -and  more  centralized-  Hierarchical  organization  in 
problem  solving. 

ELCIT:  THE  C2  EXPERIMENTATION  ENVIRONMENT 

ELICIT  is  an  open  platform  designed  and  developed  by  EBR  Inc.  and  Parity 
Communications  Inc.,  sponsored  by  the  US  DoD’s  CCRP  (Command  and  Control  Research 
Program). 
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The  version  of  ELICIT  used  for  this  research  is  the  one  described  in  Ruddy,  2007.  It  is  a 
software  based  environment  designed  to  experiment  and  research  differences  between  Edge 
and  Hierarchical  organizational  configurations  within  the  information  and  social  domains. 

It  consists  of  a  problem-solving  simulation  game,  where  players  are  organized  in  groups  of 
17  individuals  acting  as  intelligence  analysts.  In  the  game  setting,  players  interact  and 
collaborate,  within  a  networked  environment  to  accomplish  the  goal  of  identifying  a 
potential  and  fictitious  terrorist  attack.  The  purpose  is  to  complete  the  threat  recognition  by 
identifying  Who,  What,  Where  and  When  the  attack  will  occur.  During  the  game  and  across 
time,  ELICIT  provides  the  players  with  simple  -and  seemingly  independent-  pieces  of 
information  (called  “factoids”)  that  contain  clues  of  varying  value  for  accomplishing  the 
identification  goal.  Each  factoid  is  a  logical  statement  that  is  to  be  complemented  with 
other  pieces  of  information  to  build  up  situational  awareness;  however  no  participant  is 
given  sufficient  information  to  solve  his/her  problem  without  receiving  information  from 
others  (Manso  &  Nunes,  2008).  Besides,  individuals  have  to  perform  the  intelligence 
analysis,  select  the  relevant  factoids  and  share  them  to  improve  the  collective  awareness  of 
the  situation. 

The  ELICIT  environment  provides  the  option  to  manipulate  the  available  communication 
avenues  in  each  experiment,  which  enables  the  researcher  to  select  the  alternative 
organizational  form.  When  using  the  Hierarchical  structure,  the  software  randomly  points 
out  a  cross-team  coordinator  and  four  teams,  each  with  an  appointed  team  leader  and  three 
more  members.  Each  team  is  tasked  to  answer  one  of  the  solution  questions  {Who,  What, 
Where  and  When).  Under  this  configuration,  individuals  are  provided  with  factoids  related 
to  all  questions.  The  information  they  obtain  that  is  not  related  to  their  specific  question  can 
be  sent  to  players  in  other  team;  however  they  can  only  post/pull  in  the  website  associated 
to  their  team  problem.  By  contrast,  when  adopting  the  Edge  configuration  ELICIT  allows 
all  17  individuals  to  share  information  with  each  other  with  no  restriction  at  all.  The  Edge 
configuration  has  no  upper/lower  organizational  levels  or  participants  with  special 
privileges  (Manso  &  Nunes,  2008).  As  in  the  previous  organization  type,  Edge  participants 
receive  pieces  of  information  that  are  necessary  to  answer  any  of  the  four  solution 
questions. 
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The  collaboration  modes  allowed  by  ELICIT  are  “Share”,  “Post”  and  “Pull”.  The  Share 
feature  works  like  an  email  that  allows  direct  communication  among  players.  On  the  other 
hand,  Post  and  Pull  features  operate  through  websites  that  enable  information  loading  and 
downloading.  There  is  one  website  associated  to  each  question:  Who,  What,  Where  and 
When.  The  particular  functionalities  of  these  features  and  its  behavior  under  both  of  the 
organizational  forms  are  described  in  Table  N°  1. 


Collaboration 

Mode 

Description 

Edge 

Hierarchical 

Share 

Individuals  can  send  a 
factoid  to  a  specific 
player. 

All  players  can  share 
factoids  with  each 
other. 

Players  can  deliver 
factoids  to  team  mates. 
Team  leaders  can  also 
share  factoids  with  the 
cross-team  coordinator. 

Post 

Individuals  can  publish 
factoids  in  a  website 
visible  to  other  members 
of  their  entity. 

All  players  can  post 
factoids  to  every 

website. 

Players  can  only  post 
factoids  in  the  website 
associated  to  his/her 
team. 

Pull 

Individuals  can  download 
factoids  from  a  website 
accessible  to  other 

members  of  the  entity. 

Every  player  can 

download  the  posted 
factoids. 

Team  members  can 
only  access  factoids 
posted  in  the  website 
associated  to  their 
particular  team. 

Source:  adapted  from  Manso  &  Nunes,  2008. 

Table  N°  1.  Description  of  ELICIT  collaboration  features. 


BRINGING  EXPERIMENTATION  CLOSER  TO  NCW  PRACTICES 

As  indicated  before,  the  NCW  research  community  needs  to  persuade  military  leaders 
about  the  contribution  of  less  structured  organizational  practices  for  leveraging  C2 
technologies.  Accordingly,  the  main  purpose  of  experimenting  on  Edge  organizations  is  to 
develop  empirical  evidence  about  the  value  of  this  organizational  configuration  compared 
to  the  traditional  -and  currently  more  utilized-  hierarchy. 

Within  this  context,  our  analysis  of  ELICIT  functionalities  indicated  that  this  tool  exhibits  a 
limitation  with  respect  to  knowledge  sharing,  since  it  only  allows  sharing  pieces  of 
information  (i.e.  factoids);  but  not  already  analyzed  and  contextualized  notions,  which  is  an 
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essential  part  of  knowledge  conceptualization  (see  for  instance  Leonard  &  Sensiper,  1998; 
Liebowitz,  1999;  Alavi  &  Leidner,  2001) . 

We  believe  that,  in  order  to  complete  the  experimentation  environment,  it  is  necessary  at 
least  to  include  direct  communication  channels  (discussion)  and  to  emulate  the  availability 
of  intelligence  reports  from  other  entities  (elaborated  judgment).  Both  are  usual  practices  in 
real-world  C2  environment.  Combining  ELICIT  with  direct  communication  has  already 
been  explored  in  earlier  research  (see  for  instance  Leweling  &  Nissen,  2007  and  Chong  et 
al,  2008),  therefore  our  efforts  have  been  oriented  to  include  a  complementary  feature  that 
allows  each  participant  to  share  the  outcome  of  his/her  evaluation  of  the  circumstances, 
after  sharing  information  with  his/her  peers  and  thoroughly  analyzing  the  situation. 

For  selecting  this  feature,  we  have  considered  the  dimensions  through  which  C2  maturity 
level  is  measured,  so  that  our  suggested  improvements  facilitate  the  identification  of 
potential  differences  between  organizational  configurations.  We  believe  that  by 
implementing  this  new  functionality  we  are  providing  improvements  associated  with  two 
out  of  the  three  dimensions  of  the  C2  Approach  Space  (NATO  SAS-065,  2010).  The  option 
of  sharing  “threat  identifications”  (IDs)  and  knowing  what  is  the  solution  selected  by  other 
peer  analysts  at  any  moment,  offers  an  opportunity  to  develop  richer  Patters  of  Interaction 
within  the  experimentation  environment.  Besides,  providing  access  to  already  worked  out 
intelligence  benefits  Distribution  of  Information,  since  it  enhances  the  extent  to  which  the 
information  needed  to  accomplish  required  tasks  is  available. 

CIP:  COMMON  IDENTIFICATION  PICTURE  TOOL 

The  Common  Identification  Picture  Tool  (CIP)  is  a  software  tool  built  to  complement 
ELICIT  functionalities.  It  aims  to  include  in  the  experimentation  setting  the  real-world 
practice  of  issuing  intelligence  reports  and  making  them  available  to  other  units.  It  provides 
a  knowledge  sharing  avenue  to  enhance  the  information  sharing  features  already 
implemented  in  ELICIT. 

CIP  consists  of  an  application  that  allows  visualization  of  the  threat  perception  of  each 
participant  that  during  the  game  considers  has  gathered  sufficient  information  to  make  a 
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judgment  on  the  situation.  It  supports  sharing  of  complete  or  partial  attack  identifications 
(IDs),  posted  by  ELICIT  players  during  the  execution  of  experiments.  Thus,  it  enables 
participants  to  post  their  answers  to  each  of  the  Ws  (Who,  What,  Where  and  When).  The 
tool  shows  six  (6)  bar  charts  that  consolidate  the  information  regarding  each  of  the  four  Ws. 
It  deploys  three  bar  charts  for  When  as  it  divides  this  identification  into  Month,  Day  and 
Time. 

CIP  supports  both  organizational  forms  included  in  ELICIT  -Hierarchical  and  Edge¬ 
providing  similar  information  access  schemas.  For  the  Hierarchical  organization  depicted  in 
Figure  N°  1,  CIP  provides  the  following  information: 


E5 

A1...A4 

B1...B4 

C1...C4: 

D1...D4 


May  access  the  information  on  all  four  (4)  variables  (six  charts  activated). 
May  access  the  information  on  variable  “Who”  (one  chart  activated). 

May  access  the  information  on  variable  “What”  (one  chart  activated). 

May  access  the  information  on  variable  “Where”  (one  chart  activated). 
May  access  the  information  on  variable  “When”,  which  is  composed  of 
three  charts  Month,  Day  and  Time  AM/PM  (three  charts  activated). 


Figure  N°l.  Hierarchical  organization  configuration. 


For  the  Edge  organization  depicted  in  Figure  N°  2,  CIP  provides  access  to  the  information 
on  all  four  (4)  variables  (six  charts  activated). 


Figure  N°  2.  Edge  organization  configuration. 
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To  determine  the  information  access  rights  of  each  player,  CIP  requires  participants  to 
identify  themselves  through  a  username  when  they  load  the  application  at  the  beginning  of 
the  game.  This  required  username  matches  the  pseudonym  assigned  by  ELICIT,  which 
enables  the  CIP  tool  to  identify  each  player’s  role  in  the  game. 

The  general  configuration  of  the  user  interface  of  CIP  is  the  one  shown  in  Figure  N°  3.  It  is 
divided  into  three  sections:  Chart,  Identification  and  History  Section. 

The  “Chart  Section”  includes  one  chart  for  each  identification  to  be  made  as  explained 
above.  Depending  on  the  role  of  the  user  the  charts  may  be  activated  (one  grey  bar)  or 
deactivated  (one  red  bar).  The  gray  bar  in  activated  charts  indicates  the  accumulated 
number  of  pending  IDs.  Thus,  the  size  of  this  bar  diminishes  as  IDs  are  made. 


Figure  N°  3.  CIP  initial  screen  with  one  chart  activated. 


As  shown  in  Figure  N°  4,  once  the  experiment  begins  and  IDs  are  registered,  new  bars 
appear  in  the  activated  charts;  as  many  as  different  identifications  are  made.  These 
additional  bars  represent  the  accumulated  number  of  identical  IDs  already  made.  In  this 
figure  there  is  only  one  chart  activated,  meaning  that  this  is  the  screen  corresponding  to  the 
member  of  one  of  the  four  teams  of  a  Hierarchical  organization.  On  top  of  each  of  the  bars 
that  represent  a  different  ID  appears  the  number  of  participants  that  have  coincided  in  the 
same  identification.  Similarly,  on  top  of  the  gray  bar  appears  the  number  of  identifications 
pending.  When  charts  are  blocked,  the  number  on  top  of  the  red  bar  is  always  17. 
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Figure  N°  4.  CIP  initial  screen  with  one  chart  activated. 

Figure  N°  5  represents  a  situation  similar  to  that  depicted  in  Figure  N°  4;  but  the  screen  in 
this  case  corresponds  to  an  Edge  player  or  to  the  cross-team  coordinator  of  a  Hierarchical 
organization. 


Figure  N°  5.  CIP  initial  screen  with  all  six  charts  activated. 

The  identification  section  provides  fields  for  typing  in  the  IDs  corresponding  to  Who,  What 
and  Where  and  drop  down  menus  for  Month,  Day  and  Time  to  register  When  IDs.  It  also 
includes  a  “send”  and  a  “clear”  button.  Here,  participants  may  post  complete  or  partial 
identifications,  that  is,  it  is  not  necessary  to  provide  answers  for  all  of  the  questions. 
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Since  participants  can  change  their  identifications  whenever  they  want,  the  history  section 
keeps  those  identifications  that  have  been  once  registered,  but  the  author  changed  his/her 
evaluation  afterwards.  This  record  is  cleared  when  a  new  experiment  is  started. 

Each  time  an  identification  is  registered,  the  charts  are  updated  to  reflect  the  new  entry. 

HYPOTHESIS 

Since  the  CIP  tool  enhances  the  functionalities  associated  to  the  execution  of  advanced 
Patterns  of  Interaction  and  improved  Distribution  of  Information  among  entities  (NATO 
SAS-065,  2010),  we  believe  that  our  experiment  results  will  evidence  better  unit  problem¬ 
solving  performance  for  entities  utilizing  this  tool,  while  the  organizational  configuration  is 
kept  constant.  We  also  expect  that  Edge  organizations  outperform  Hierarchical  ones  both 
using  and  without  using  the  CIP  tool  since,  as  indicated  in  the  associated  theory  (Alberts  & 
Hayes,  2003;  2006),  this  outcome  is  a  generally  expected  consequence  of  working  in  a 
networked  environment  such  as  the  one  provided  by  ELICIT.  Further,  by  combining  these 
two  conceptions,  we  also  conjecture  that  the  difference  in  the  problem-solving  performance 
of  a  Hierarchical  organization  using  CIP  and  an  Edge  organization  without  using  such 
instrument  will  be  rather  meager. 

Given  the  arguments  above,  we  set  the  following  hypotheses: 

Hypothesis  A:  The  problem-solving  performance  of  the  Edge  organization  using  CIP  will 
be  superior  to  that  of  the  Edge  organization  without  using  CIP. 

Hypothesis  B:  The  problem-solving  performance  of  the  Hierarchical  organization  using 
CIP  will  be  superior  to  that  of  the  Hierarchical  organization  without  using 
CIP. 

Hypothesis  C:  The  problem-solving  performance  of  the  Edge  organization  without  using 
CIP  will  be  superior  to  that  of  the  Hierarchical  organization  without  using 
CIP. 

Hypothesis  D:  The  problem-solving  performance  of  the  Edge  organization  using  CIP  will 
be  superior  to  that  of  the  Hierarchical  organization  using  CIP. 
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Hypothesis  E:  The  problem-solving  performance  of  the  Edge  organization  without  using 
CIP  will  be  similar  to  that  of  the  Hierarchical  organization  using  CIP. 

Even  though  we  will  call  our  propositions  “hypothesis”,  we  warn  the  reader  that  our 
experiments  and  their  results  will  not  be  sufficient  to  validate  or  reject  strict  hypothesis, 
given  the  reduced  sample  size  (17  participants  in  each  of  4  groups),  which  is  insufficient  for 
proper  and  complete  statistical  analysis. 

EXPERIMENTATION  METHODOLOGY 

Experiment  Design 

Our  set  of  experiments  comprised  four  ELICIT  runs,  each  with  a  different  group  of 
participants. 

Every  group  used  the  ELICIT  platform,  two  of  them  adopted  an  Edge  configuration  and  the 
other  two  a  Hierarchical  structure.  Besides,  one  of  the  Edge  and  one  of  the  Hierarchical 
groups  used  the  CIP  (Common  Identification  Picture)  tool  to  complement  the  ELICIT 
platform.  Therefore  the  groups  participating  in  each  of  the  experiments  were  the  following 
(see  Figure  N°6): 

-  Group  N°  1 :  Edge  NO  CIP 

-  Group  N°2:  Edge  USING  CIP 

-  Group  N°3:  Hierarchical  NO  CIP 

-  Group  N°4:  Hierarchical  USING  CIP 


Figure  N°6.  Groups  and  hypotheses. 
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The  combination  and  comparison  of  data  between  these  groups  gave  rise  to  our 
experiments  and  their  associated  hypotheses: 

-  Experiment  A:  Comparison  between  two  Edge  groups,  one  of  them  used  CIP  tool. 

-  Experiment  B:  Comparison  between  two  Hierarchical  groups,  one  of  them  used  CIP 
tool. 

-  Experiment  C:  Comparison  between  two  groups,  Edge  and  Hierarchical  respectively. 
None  of  them  used  CIP  tool. 

-  Experiment  D:  Comparison  between  two  groups,  Edge  and  Hierarchical  respectively. 
Both  of  them  used  CIP  tool. 

-  Experiment  E:  Comparison  between  two  groups,  one  Hierarchical  using  CIP  tool  and 
one  Edge  without  using  such  instrument. 

Participants 

The  experiments  involved  68  Army  officers  from  the  Military  Polytechnic  Academy  of  the 
Chilean  Army.  They  were  mostly  Lieutenants  and  Captains  with  an  average  of  8  years  of 
professional  experience.  To  run  the  experiments,  they  were  organized  into  the  four  groups 
identified  above.  Each  group  was  internally  as  homogeneous  as  possible  and  externally 
equivalent  to  the  other  groups  in  the  averaging  background  knowledge  and  experience  of 
the  assigned  participants  as  well  as  in  gender,  age  and  military  ranks. 

Prior  to  the  experiment  every  group  attended  a  standardized  instructional  session  where 
they  watched  the  ELICIT  video  (translated  to  Spanish)  and  were  given  a  presentation  on  the 
experiment  purpose,  ELICIT  software  features  and,  when  necessary,  CIP  functionalities 
and  operation.  After  the  theoretical  preparation,  all  groups  went  through  a  practical  training 
run.  The  information  used  for  training  purposes  was  different  from  that  used  in  the  actual 
experiments. 

Controls 

In  order  to  ensure  that  each  run  was  played  with  totally  unknown  information,  four  new 
factoid  sets  were  prepared  and  used  in  our  experimental  sessions.  All  factoid  changes  were 
not  structural  but  only  superficial  (e.g.  Alphaland  became  Thetaland).  Besides,  to  keep 
steady  conditions  either  instructions  and  training  or  software  configurations;  as  well  as 
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physical  laboratory  environments  remained  unchanged  across  experiment  runs.  Further, 
participants  were  instructed  not  to  communicate  with  each  other  except  via  the  ELICIT  or 
CIP  applications. 

Variables 

The  independent  variable  was  either  “Organizational  Configuration”  (Edge  or  Hierarchy)  or 
“Usage  of  CIP  Tool”.  The  former  follows  the  original  ELICIT  experimentation  design, 
whereas  the  latter  is  conceptualized  as  an  enhancement  of  the  experimentation  environment 
with  a  knowledge  sharing  channel  that  facilitate  improving  both  the  patterns  of  interactions 
and  the  distribution  of  information  among  entities.  Since  both  of  the  independent  variables 
are  binary,  a  total  of  four  groups  -as  described  above-  covered  the  desired  experimental 
conditions. 

In  every  case  the  dependent  variable  was  “Performance”  (P),  which  should  be  understood 
as  the  ability  of  an  organization  of  making  decisions  based  on  correct  shared  awareness  of  a 
circumstantial  situation  gained  in  a  minimum  time. 

Specifically,  the  variable  P  was  defined  as  being  directly  proportional  to  “Accuracy”  (A), 
and  inversely  proportional  to  Time  (T).  In  other  words,  P  increases  as  A  increases  and 
decreases  as  T  increases. 


P  =  K*  (A/T) 


Where  K  =  100,000,  a  constant  included  just  for  convenience  in  calculations  and  readability 
of  figures. 

The  Performance  (P)  of  each  group  was  obtained  by  calculating  the  average  of  the 
performance  of  all  the  members  of  the  group. 

The  variable  “Accuracy”  (A)  measured  the  ability  of  organizations  to  gain  correct  shared 
awareness  of  the  situation  under  analysis.  To  obtain  the  value  of  this  variable,  a  particular 
score  was  assigned  to  each  correct  answer  of  every  team  member.  Correct  answers  to 
questions  Who,  What  and  Where  were  assigned  0.25  points  each,  whereas  question  When 
was  divided  into  three  portions:  month,  day  and  time,  weighing  0.083  each.  No  score  (or 


-  14- 


Testing  Edge  versus  Hierarchy  C2  Organizations  using 
the  ELICIT  platform  and  Common  Identification  Picture  tool 


zero)  was  assigned  to  wrong  or  missing  answers.  The  result  for  each  participant  was 
computed  as  the  addition  of  his/her  scores.  Therefore,  the  highest  individual  score  was  1.0 
if  all  answers  were  right.  Finally,  the  Accuracy  of  a  group  was  calculated  as  the  average  of 
the  sores  of  all  its  members. 

The  variable  “Time”  (T)  measured  the  total  time  taken  by  participants  to  submit  a  threat 
identification  through  the  ELICIT  platform.  Accordingly,  the  figures  used  in  our 
calculations  were  obtained  from  the  log  files  of  the  ELICIT  software. 

Data  Analysis 

Data  analyses  were  performed  using  the  statistical  package  SPSS  13.0  (www.spss.comf.  A 
detailed  description  of  the  computations  and  its  results  are  included  in  Appendix  1 . 

Once  the  figures  were  extracted  from  the  log  files,  the  individual  Performance  (P)  was 
computed.  For  each  hypothesis  we  followed  two  steps: 

First,  we  obtained  a  comparative  boxplot  in  order  to  examine  and  contrast  the  main 
parameters  of  the  data  distributions. 

Second,  we  run  an  ANOVA  hypothesis  test  using  95%  of  confidence  (a  =  0.05).  For  this 
purpose,  we  first  checked  for  compliance  of  the  conditions  that  validate  the  appropriateness 
of  applying  an  ANOVA  test  (Petrie  &  Watson,  2006): 

-  We  checked  for  normality  through  the  Kolmogorov-Smimov  and  the  Shapiro-Wilk 
tests. 

-  We  also  verified  homogeneity  of  variances  through  the  Levene  test. 

After  compliance  was  confirmed,  we  performed  a  mean  comparison  through  ANOVA  and 
-whenever  possible-  rejected  the  corresponding  null  hypothesis  within  a  confidence 
interval  of  95%. 

If  the  results  of  the  tests  mentioned  above  were  not  satisfactory,  and  therefore  ANOVA  was 
not  applicable,  we  applied  the  non  parametric  Kruskal-Wallis  Test  for  mean  comparison,  as 
recommended  in  Camacho,  2006. 
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These  procedures  gave  us  sufficient  information  to  compare  the  “Performance”  of  the 
groups  involved  in  every  hypothesis,  which  enabled  us  to  validate  or  reject  such 
propositions. 

In  order  to  check  if  complementary  information  could  be  captured  from  the  communication 
behavior  among  entities  during  the  experiments,  we  also  examined  and  practiced  ANOVA 
tests  using  the  figures  of  “Share”,  “Post”  and  “Pull”  variables  provided  by  the  ELICIT  log 
files. 


RESULTS 

As  described  in  Appendix  1,  our  normality  checks  for  the  dependent  variable 
“Performance”,  executed  prior  to  all  hypotheses  testing,  yielded  satisfactory  results.  Similar 
outcomes  were  obtained  for  homogeneity  of  variances  check  in  every  Levene  test  we 
practiced.  Normal  distribution  of  and  similarity  of  variances  between  the  dependent 
variables  support  the  suitability  of  applying  ANOVA  to  determine  if  the  manipulations  of 
the  independent  variables  resulted  in  statistically  significant  differences  among  the  sample 
populations.  The  only  exception  to  this  assertion  was  found  in  the  testing  of  Hypothesis  E, 
where  ANOVA  was  not  suitable  and  therefore  a  Kruskal-Wallis  Test  was  applied. 


CIP  usage 

When  manipulating  CIP  usage  combined  with  Edge  organizational  form,  we  obtained 
figures  indicating  that  the  CIP  usage  have  a  significant  positive  effect  (p  =  0.044;  see 
Appendix  1).  To  explain  this  result,  we  observe  that  the  differences  between  the  means  of 
Performance  component  factors  varied  considerably.  While  the  means  of  Time  were  not 
very  different  (1323  for  EDGE  NO  CIP  and  1622  for  EDGE  USING  CIP),  the  means  of 
Accuracy  were  notably  dissimilar  (47%  for  EDGE  NO  CIP  and  86%  for  EDGE  USING 
CIP),  which  finally  accounts  for  a  superior  mean  of  Performance  variable  of  the  group 
using  CIP  (38.42  for  EDGE  NO  CIP  versus  54.69  for  EDGE  USING  CIP).  Therefore  we 
attribute  the  improved  performance  observed  when  using  CIP  mainly  to  a  better  shared 
situational  awareness.  This  result  validates  our  Hypothesis  A,  therefore  ‘The  problem- 
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solving  performance  of  the  Edge  organization  using  CIP  was  superior  to  that  of  the 
Edge  organization  without  using  CIP”. 

Similarly,  when  varying  CIP  usage  while  keeping  Hierarchical  configuration  unchanged, 
we  also  found  a  significant  positive  effect  of  CIP  usage  {p  =  0.003;  see  Appendix  1).  Here, 
again,  the  means  of  the  Time  variable  were  rather  alike  (1734  for  HIERARCHY  NO  CIP 
and  1950  for  HIERARCHY  USING  CIP);  yet  the  means  of  Accuracy  were  quite  different 
(38.73%  for  HIERARCHY  NO  CIP  and  75.49%  for  HIERARCHY  USING  CIP). 
Therefore,  Accuracy  is  the  factor  that  explains  the  higher  Performance  mean  obtained  by 
the  organization  using  CIP  (22.35  for  HIERARCHY  NO  CIP  versus  39.55  for 
HIERARCHY  USING  CIP).  Accordingly,  in  this  additional  case  we  once  more  attribute 
this  enhanced  performance  mainly  to  a  better  shared  situational  awareness  achieved 
through  CIP  usage.  This  result  validates  our  Hypothesis  B,  therefore  ‘The  problem¬ 
solving  performance  of  the  Hierarchical  organization  using  CIP  was  superior  to  that  of 
the  Hierarchical  organization  without  using  CIP”. 

It  is  important  to  note  that  in  both  cases  analyzed  above  the  mean  of  the  Time  variable  in 
the  groups  that  did  not  use  CIP  was  a  little  lower  than  the  mean  of  the  group  using  CIP,  in 
other  words,  CIP  usage  slightly  slows  down  the  decision  making  process,  however  it  still 
yields  a  positive  net  effect  in  Performance  because  of  the  much  better  Accuracy  it  brings 
about. 


Organizational  configuration 

When  we  contrasted  the  Performance  of  Edge  versus  Hierarchical  organizational  form 
using  just  the  ELICIT  platform,  our  results  indicated  that  there  is  no  significant  difference 
in  the  performance  of  both  kinds  of  organization  (p  =  0.051,  see  Appendix  1).  In  order  to 
explain  this  result  we  examined  the  means  of  the  factors  of  Performance  variable.  Within 
this  context,  it  becomes  evident  that  the  Edge  outperforms  the  Hierarchical  organization; 
however,  the  differences  are  reduced.  While  the  Time  means  indicate  that  the  Edge 
organization  was  in  average  23.7%  faster  than  the  Hierarchy  (1323  for  EDGE  NO  CIP  and 
1734  for  HIERARCHY  NO  CIP),  the  Accuracy  means  render  a  much  tighter  difference  of 
11.73%  in  favor  of  the  Edge  configuration  (47%  for  EDGE  NO  CIP  and  38.73%  for 
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HIERARCHY  NO  CIP).  Thus,  our  results  provide  some  hints  that  allow  us  to  conjecture 
about  a  superior  performance  of  the  Edge  organization;  however  our  outcomes  are  not 
conclusive.  These  arguments  induce  us  to  reject  our  Hypothesis  C;  therefore  “The 
problem-solving  performance  of  the  Edge  organization  without  using  CIP  was  not 
superior  to  that  of  the  Hierarchical  organization  without  using  CIP”. 

By  contrast,  when  evaluating  the  Performance  of  both  kinds  of  organizational 
configurations  while  consistently  using  the  CIP  tool,  the  results  clearly  indicated  that  the 
Edge  organization  performs  better  than  the  Hierarchy  (p  =  0.006;  see  Appendix  1).  These 
results  are  confirmed  by  the  analysis  of  the  means  of  Performance  factors.  The  comparison 
of  the  Time  means  indicates  that  the  Edge  organization  is  16.8  %  faster  than  the  Hierarchy 
(1622  for  EDGE  USING  CIP  versus  1950  for  HIERARCHY  USING  CIP),  whereas  the 
means  of  Accuracy  indicate  that  the  -correct-  perception  of  the  situation  in  the  Edge 
organization  is  also  higher  than  in  the  Hierarchy  (86%  for  EDGE  USING  CIP  and  75.49% 
for  HIERARCHY  USING  CIP).  It  is  relevant  to  notice  that  in  this  case  the  results  of  both 
factors  are  better  for  the  Edge  organization.  In  other  words,  the  Edge  organization  is  both 
faster  and  more  accurate  than  the  Hierarchy.  Thus,  the  superior  results  for  Performance 
variable  exhibited  by  the  Edge  organization  is  a  consequence  of  the  results  of  both  of  its 
factors.  These  results  validate  our  Hypothesis  D;  therefore,  “The  problem-solving 
performance  of  the  Edge  organization  using  CIP  was  superior  to  that  of  the 
Hierarchical  organization  using  CIP”. 


CIP  usage  and  organizational  configuration 

We  further  evaluated  our  data  to  compare  Performance  if  both  CIP  usage  and 
organizational  configuration  were  varied.  In  this  context,  when  comparing  the  figures  of  the 
Edge  organization  without  using  CIP  with  the  Hierarchical  organization  using  the  tool,  we 
found  similar  performance.  In  fact,  our  statistical  comparison  between  the  performance  of 
these  two  groups  yielded  no  statistically  significant  difference  (p  =  0.513,  see  Appendix  1). 
We  attribute  this  similarity  to  the  fact  that  one  of  the  factors  of  the  variable  Performance 
favors  the  results  of  the  first  group,  while  the  other  favors  the  outcomes  of  the  second  one. 
Specifically,  Time  means  indicate  that  the  Edge  organization  was  in  average  32%  faster 
than  the  Hierarchy  (1323  for  EDGE  NO  CIP  and  1950  for  HIERARCHY  USING  CIP).  By 
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contrast,  the  Accuracy  means  indicate  that  the  Hierarchy  is  28.49%  more  accurate  than  the 
Edge  organization  (47%  for  EDGE  NO  CIP  and  75.49%  for  HIERARCHY  USING  CIP). 
These  opposite  results  make  the  Performance  means  of  both  groups  look  very  alike,  even 
though  the  populations  are  completely  different.  From  these  result  observations  we  can 
validate  our  Hypothesis  E;  therefore  we  state  that  “The  problem-solving  performance  of 
the  Edge  organization  without  using  CIP  was  similar  to  that  of  the  Hierarchical 
organization  using  CIP”. 

Finally,  our  analyses  of  the  data  provided  by  the  ELICIT  log  files  regarding  Share,  Post  and 
Pull  variables  did  not  evidence  statistically  significant  differences  between  the 
communication  behaviors  of  any  of  the  groups  tested  in  these  experiments. 


RESULT  DISCUSION 

As  expected,  the  results  of  our  experiments  clearly  indicate  that  the  introduction  of  the  CIP 
tool,  to  complement  the  functionalities  provided  by  ELICIT,  improved  the  performance  of 
all  types  of  organizations  operating  in  a  networked  environment.  This  outcome  is  explained 
by  the  fact  that  the  CIP  tool  provided  an  inexistent  avenue  that  allowed  sharing  already 
analyzed  and  contextualized  knowledge.  This  functionality  accompanied  by  the  previously 
available  information  sharing  channels  provides  a  more  complete  -and  also  realistic- 
setting  for  decision-making  ability  testing.  This  conjoint  set  of  tools  resembles  the 
interaction  channels  accessible  for  tactical  command  posts,  where  messaging  and  sensors 
provide  intelligence  clues  and  also  some  official  reports  from  higher  command  and  from 
parallel  and  lower  level  units. 

Besides,  from  a  theoretical  standpoint,  we  contend  that  the  CIP  tool  helps  organizations  to 
enhance  their  abilities  in  two  of  the  dimensions  in  which  C2  maturity  level  is  weighed 
(NATO  SAS-065,  2010),  namely  Patterns  of  Interactions  and  Distribution  of  Information. 
By  sharing  decisions  already  worked  out,  organizations  interact  at  the  cognitive  level  -as 
opposed  to  the  less  elaborated  and  more  constrained  information  level-  which  yields 
opportunities  for  rising  interactions  to  a  more  refined  and  less  intellectually  restricted  level 
that,  in  turn,  allows  for  more  unconstrained  interaction  patterns.  We  believe  these 
arguments  neatly  reflect  the  spirit  of  the  definition  provided  in  NATO  for  the  former 
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dimension:  “Patterns  of  interaction  among  participant  entities  are  a  function  of  their 
respective  abilities  and  willingness  to  interact  as  well  as  the  opportunities  they  have  as  a 
result  of  the  actual  occurrence  of  interactions  and  collaboration”  (NATO  SAS-065,  2010 
p.  48). 

Additionally,  when  the  distributed  product  is  both  information  and  knowledge  derived  from 
that  information,  the  receiver  obtains  an  extra  value-added  set  of  notions  that  implies  access 
to  a  broader  and  more  integrated  body  of  ideas  that  facilitate  decision  making.  Here,  again, 
we  argue  that  this  explanation  captures  the  essence  of  the  definition  provided  by  NATO  for 
the  latter  dimension:  “Distribution  of  information  across  participating  entities  refers  to  the 
extent  to  which  the  information  needed  to  accomplish  required  tasks  is  available  to  each 
participant”  (NATO  SAS-065,  2010  p.  49). 

Expected  as  well  was  the  fact  that,  regardless  of  the  organizational  configuration,  it  took  a 
little  more  time  to  make  decisions  when  using  the  CIP  tool.  This  result  is  a  consequence  of 
having  a  more  complete  set  of  interaction  channels.  Yet,  it  was  less  expected  that  this 
slower  decision  process  would  considerably  increment  the  accuracy  of  such  judgments.  We 
had  anticipated  a  better  accuracy;  however  the  magnitude  of  the  improvement  was  rather 
unforeseen.  We  attribute  this  significant  improvement  to  the  fact  that  providing  information 
apprises  the  receiver,  who  still  needs  to  perform  analysis  to  understand  the  context  and  if 
time  is  constrained,  decisions  turn  out  less  precise.  Sharing  knowledge,  on  the  other  hand, 
offers  access  to  notions  that  already  have  the  potential  to  influence  action  (Alavi  &  Leidner, 
2001),  therefore  the  receiver  efforts  can  concentrate  on  detailed  analysis  that  add  precision 
to  his/her  decisions. 

More  surprising  was  the  result  indicating  that  when  comparing  the  performance  of  both 
organizational  forms,  there  was  scarce  discrepancy  when  CIP  was  not  used;  but 
considerable  difference  when  this  tool  was  utilized.  We  attribute  the  difference  in  the 
“Time”  variable  to  the  fact  that  the  Edge  organization  enable  concurrent  and  dynamic 
information  processing  whereas  Hierarchies  follow  sequential  processing  through  the  chain 
of  command,  which  slows  down  decision-making  (Alberts  and  Hayes,  2003).  From  a  more 
general  perspective,  we  contend  that  this  outcome  indicates  that  incremented  interaction 
channels  as  well  as  increased  cognitive  level  teamwork,  benefits  the  less  constrained 


-20- 


Testing  Edge  versus  Hierarchy  C2  Organizations  using 
the  ELICIT  platform  and  Common  Identification  Picture  tool 


organization  in  a  higher  degree.  Specifically,  we  argue  that  the  enrichment  of  the 
interaction  space  was  better  exploited  by  the  kind  of  organization  featuring  a  more 
complete  pattern  of  interactions  and  freedom  to  collaborate  as  opposed  to  the  more 
restricted  one.  Put  in  other  words,  we  contend  that  the  organization  type  that  is  better  able 
to  exploit  the  new  functionality  reaped  more  benefits  out  of  the  improvements  provided  by 
the  CIP  tool. 

As  a  final  point,  we  contend  that  as  a  consequence  of  improving  the  interaction  channels 
and  raising  the  cognitive  level  of  collaboration  efforts,  the  hierarchical  organization  can 
mitigate  its  structural  communicational  limitations,  making  its  problem  solving 
performance  equivalent  to  that  of  an  Edge  organization  in  which  such  improvements  have 
not  been  made. 


CONCLUSIONS 

In  this  paper  we  have  proposed  a  complementary  tool  that  enriches  the  experimentation 
environment  provided  by  the  ELICIT  platform,  which  we  term  Common  Identification 
Picture  (CIP)  tool.  After  presenting  this  application,  we  explained  the  theoretical 
foundations  backing  its  development  and  the  way  it  complements  the  ELICIT  platform. 
Subsequently,  we  formulated  five  hypotheses  aimed  at  verifying  the  effects  and  usefulness 
of  the  CIP  tool.  We  then  tested  our  theorized  suggestions  through  a  set  of  five  carefully 
designed  experiments  that  were  implemented  at  the  Polytechnic  Academy  of  the  Chilean 
Army. 

The  CIP  tool  allows  the  players  to  share  their  understanding  of  the  situation  after  analyzing 
the  pieces  of  information  provided  by  ELICIT.  It  resembles  the  widespread  practice  of 
writing  and  sharing  intelligence  reports  among  units  in  the  field,  which  enable  the 
recipients  to  assemble  their  own  situational  picture.  In  summary,  this  application  facilitates 
knowledge  sharing  among  operational  entities. 

Our  results  indicate  that,  when  units  make  efforts  to  communicate  their  understanding  of 
the  situation  to  their  peers,  it  becomes  easier  for  the  collective  to  form  “correct”  shared 
awareness.  Such  practice  might  take  a  little  longer  than  a  process  in  which  sharing 
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knowledge  is  excluded;  nevertheless  the  added  accuracy  considerably  pays  off  the 
additional  time  consumed.  These  outcomes  suggest  that,  in  order  to  leverage  the  new 
functionalities  provided  by  a  networked  setting,  the  practice  of  performing  information 
analysis,  and  diffusing/sharing  intelligence  at  all  levels  should  be  stressed  and  deeply 
embedded  in  doctrinal  practices. 

Our  comparison  of  different  organizational  structures  reveals  that  the  less  structured  entities 
(i.e.  Edge)  are  better  able  to  take  advantage  of  knowledge  sharing  functionalities  than  the 
more  restricted  ones  (i.e.  Hierarchies),  since  the  former  enjoys  more  complete  pattern  of 
interactions  and  enhanced  freedom  to  collaborate.  Interestingly,  the  noticeable  difference 
among  both  kinds  of  organizations  emerges  only  when  knowledge  sharing  is  implemented. 
These  results  led  us  to  make  two  conclusions. 

First,  our  empirical  work  supports  the  theoretical  work  regarding  NNC2MM  (NATO  SAS- 
065,  2010)  and  “ Power  to  the  Edge ”  (Alberts  &  Hayes,  2003),  since  it  provides  evidence 
indicating  that  in  a  NCW  environment  the  Edge  organization  can  exhibit  superior 
performance  than  Hierarchies.  Therefore,  it  is  correct  to  recommend  Edge  configuration 
usage  whenever  the  situation  permits  lowering  the  level  of  decision  right  allocation.  And 
second,  in  order  to  materialize  Edge  superior  performance,  all  the  potential  of  NCW 
environment  must  be  implemented  and  exploited.  This  includes,  but  is  not  restricted  to,  rich 
communication  channels;  information  diffusion  and,  indeed,  knowledge  sharing. 

Since  sometimes  it  is  not  possible  to  adopt  an  Edge  configuration,  it  is  relevant  to  note  that 
the  architectural  communication  restrictions  of  Hierarchies  can  be  reduced  by  providing 
technology  and  implementing  procedures  that  enable  and  encourage  intelligence  sharing. 
Such  enhanced  interaction  capability  may  allow  Hierarchies  to  improve  their  ability  to 
achieve  shared  awareness  and  to  perform  at  levels  very  similar  to  those  of  Edge 
organizations  that  only  share  information. 

Based  on  our  findings,  we  recommend  enhancing  the  functionalities  of  the  ELICIT 
platform  so  that  it  facilitates  more  complete  interactions  and  fosters  knowledge  sharing 
among  entities.  Such  enhancements  should  enable  measurements  of  variables  associated  to 
“ Agility ”  attributes  (Alberts  &  Hayes,  2003,  p.  128),  which  are  not  collected  by  the  current 
version. 
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Although  our  results  clearly  back  our  conclusions  above,  we  would  like  to  warn  the  reader 
that  these  outcomes  are  far  from  being  conclusive,  as  the  reliability  and  validity  of  the 
outcomes  can  not  be  accredited  through  the  reduced  number  of  observations  that  we  were 
able  to  capture.  However,  these  results  combined  with  those  of  a  number  of  earlier 
experiments  carried  out  within  the  ELICIT  scientific  community  may  validate  part  of  our 
findings,  namely  those  that  surface  the  differences  between  Edge  and  Hierarchical 
organizations. 

Future  work  should  be  oriented  to  verify  the  robustness  of  these  results  by  implementing 
similar  experiments  that  complete  the  sample  size.  For  this  purpose,  the  Military 
Polytechnic  Academy  of  the  Chilean  Army  is  open  to  share  the  CIP  tool  with  members  of 
the  ELICIT  community. 
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APPENDIX  1 

EXPERIMENT  RESULTS 


A.  TESTING  OF  HYPOTHESIS  “A”  (H-A) 

1.  COMPARING:  EDGE  NO  CIP  -  EDGE  USING  CIP 


Dependent  Variable 
Independent  Variable: 
Organization  Type: 
Group  1: 

Group  2: 


P  =  Group  Performance 

CIP  tool  usage  (Common  Identification  Picture  tool  usage) 
EDGE 

EDGE  NO  CIP 
EDGE  USING  CIP 


a.  Performance  variable  distribution. 


The  boxplot  above  shows  a  graphical  representation  of  the  distribution  data.  It  is 
evident  that  the  “Edge  Using  CIP”  group  (Group  2)  clearly  outperformed  the  “Edge 
No  CIP”  group  (Group  1). 
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From  the  boxplot,  it  can  also  be  observed  that  the  overall  data  distribution  of  Group 
2  spans  through  higher  values  than  the  distribution  data  of  Group  1 . 

The  most  relevant  parameters  of  the  distribution  data  are  the  following: 


Fferformance 


N 

IVtean 

Std.  Deviation 

Std.  Era 

95%  Confidence  Interval  for 

IVfean 

Mnirrum 

IVbxirrum 

Lower  Bound 

Upper  Bound 

Edge  NO  OP 

17 

38,4188 

27,28475 

6,61752 

24,3903 

52,4473 

,00 

111,36 

Edge  USING  OP 

17 

54,6935 

16,66355 

4,04150 

46,1259 

63,2611 

12,68 

74,35 

Total 

34 

46,5562 

23,74452 

4,07215 

38,2713 

54,8410 

,00 

111,36 

Table  N°  2.  Relevant  parameters  of  Performance  distribution,  H-A 


b.  Compliance  of  conditions  for  ANOVA  hypothesis  test,  Performance  variable, 
H-A. 

We  practiced  the  Kolmogorov-Smimov  and  the  Shapiro-Wilk  tests  for  normality 
check.  We  also  verified  homogeneity  of  variances  through  the  Levene  test.  For  all 
tests  we  used  95%  of  confidence  (a  =  0.05).  Our  results  turned  out  satisfactory  as 
indicated  in  the  following  tables. 


Tests  of  Normality 


GROUP 

Kolmogorov-Smirnov3 

Shapiro-Wilk  | 

Statistic 

df 

Sig. 

Statistic 

df 

Sig.  ! 

Perf.  N°  1 

,132 

17 

,927 

17 

N°  2 

,131 

17 

,919 

17 

*.  This  is  a  lower  bound  of  the  true  significance, 
a.  Lilliefors  Significance  Correction 


Table  N°  3.  Test  of  normality  results,  H-A. 

Since  for  both  groups  Sig.  (Significance  or  P  value)  is  greater  than  a  =  0.05  in  both 
tests,  then  the  normality  of  the  distribution  is  confirmed. 

Test  of  Homogeneity  of  Variances 


PERF 


Levene 

Statistic 

dfl 

df2 

Sig. 

3,084 

1 

32 

,089 

Table  N°  4.  Test  of  homogeneity  of  variance  results,  H-A 

The  value  of  the  parameter  Sig  =  0.089  is  greater  than  a  =  0.05.  This  verifies  the 
homogeneity  of  variance  of  the  distribution. 
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c.  ANOVA  results  for  mean  comparison  of  Performance  variable,  H-A. 

In  this  section  we  check  whether  the  mean  values  of  Performance  variable  of  both 
groups  differ  significantly  as  a  consequence  of  using  the  CIP  tool. 

Null  Hypothesis  “Ho”:  The  means  of  the  distribution  of  Performance  variable  in 
both  groups  are  equal  (similar). 


ANOVA  results: 


ANOVA 

PERF 


Sum  of 
Squares 

df 

Mean  Square 

F 

Sig. 

Between  Groups 

2251,361 

1 

2251,361 

4,405 

,044 

Within  Groups 

16354,106 

32 

511,066 

Total 

18605,467 

33 

Table  N°  5.  ANOVA  results  for  Performance  variable,  H-A 


Since  Sig  =  0.044  is  lower  than  a  =  0.05,  the  Null  Hypothesis  “Ho”  must  be 
rejected,  which  means  that  there  exists  a  difference  between  the  means  of  both 
distributions  at  this  significance  level. 


We  state  that  the  performance  of  Group  2  EDGE  USING  CIP  is  significantly 
superior  to  the  performance  of  Group  1  EDGE  NO  CIP,  within  a  confidence 
interval  of  95%,  which  is  legitimately  attributable  to  the  use  of  CIP. 

The  following  is  a  result  summary  for  Performance  variable: 


PERFORMANCE 

EDGE 

NO  CIP 

EDGE 
USING  CIP 

Hypothesis  Test 

Confidence 

Interval 

Mean 

38.42 

54.69 

There  exists 
significant 
difference 

95% 

Standard  Deviation 

27.28 

16.66 

Comparison  variable:  CIP  Usage 

Table  N°  6.  Results  summary  for  Performance  variable,  H-A. 


d.  ANOVA  results  for  mean  comparison  Share,  Post  and  Pull  variables,  H-A. 

We  practiced  an  ANOVA  procedure  to  compare  the  means  of  the  variables  Share, 
Post  and  Pull  of  Groups  1  and  2;  however  no  significant  differences  were  detected. 


-27- 


Testing  Edge  versus  Hierarchy  C2  Organizations  using 
the  ELICIT  platform  and  Common  Identification  Picture  tool 


B.  TESTING  OF  HYPOTHESIS  “B”  (H-B) 

1.  HIERARCHY  NO  CIP  -  HIERARCHY  USING  CIP 


Dependent  Variable 
Independent  Variable: 
Organization  Type: 
Group  3: 

Group  4: 


P  =  Group  Performance 

CIP  tool  usage  (Common  Identification  Picture  tool  usage) 

HIERARCHY 

HIERARCHY  NO  CIP 

HIERARCHY  USING  CIP 


a.  Performance  variable  distribution. 


The  boxplot  above  shows  a  graphical  representation  of  the  distribution  data  of  both 
groups.  It  is  evident  that  the  “Hierarchy  Using  CIP”  group  (Group  4)  clearly 
outperformed  the  “Hierarchy  No  CIP”  group  (Group  3). 

From  the  boxplot,  it  can  also  be  observed  that  the  overall  data  distribution  of  Group 
4  spans  through  higher  values  than  the  distribution  data  of  Group  3. 
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The  most  relevant  parameters  of  the  distribution  data  are  the  following: 


Fferformance 


N 

IVfean 

Std.  Deviation 

Std.  Error 

95%  Confidence  Interval  for 

IVfean 

Mnirrum 

IVfexirrum 

Lower  Bound 

Upper  Bound 

hierarchy  NOQP 

17 

22,3524 

17,98712 

4,36252 

13,1042 

31,6005 

,00 

63,09 

Herarchy  USING  OP 

17 

39,5500 

12,96281 

3,14394 

32,8851 

46,2149 

,00 

57,60 

Total 

34 

30,9512 

17,73465 

3,04147 

24,7633 

37,1391 

,00 

63,09 

Table  N°  7.  Relevant  parameters  of  Performance  distribution,  H-B 


b.  Compliance  of  conditions  for  ANOVA  hypothesis  test,  Performance  variable, 
H-B. 

We  practiced  the  Kolmogorov-Smimov  and  the  Shapiro-Wilk  tests  for  normality 
check.  We  also  verified  homogeneity  of  variances  through  the  Levene  test.  For  all 
tests  we  used  95%  of  confidence  (a  =  0.05).  Our  results  turned  out  satisfactory  as 
indicated  in  the  following  tables. 


Tests  of  Normality 


GROUP 

Kolmogorov-Smirnov3 

Shapiro-Wilk 

Statistic 

df 

Sig. 

Statistic 

df 

Sig. 

Perf.  N°  3 

,165 

17 

,935 

N°  4 

,209 

17 

■H 

,851 

*.  This  is  a  lower  bound  of  the  true  significance, 
a.  Lilliefors  Significance  Correction 


Table  N°  8.  Test  of  normality  results,  H-B. 

Since  for  both  groups  Sig.  (Significance  or  P  value)  is  greater  than  a  =  0.05  in  both 
tests,  then  the  normality  of  the  distribution  is  confirmed. 


Test  of  Homogeneity  of  Variances 

PERF 


Levene 

Statistic 

dfl 

df2 

Sig. 

3,041 

1 

32 

,091 

Table  N°  9.  Test  of  homogeneity  of  variance  results,  H-B 

The  value  of  the  parameter  Sig  =  0.091  is  greater  than  a  =  0.05.  This  verifies  the 
homogeneity  of  variance  of  the  distribution. 
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c.  ANOVA  results  for  mean  comparison  of  Performance  variable,  H-B. 

In  this  section  we  check  whether  the  mean  values  of  Performance  variable  of  both 
groups  differ  significantly  as  a  consequence  of  using  the  CIP  tool. 

Null  Hypothesis  “Ho”:  The  means  of  the  distribution  of  Performance  variable  in 
both  groups  are  equal  (similar). 

ANOVA  results: 


ANOVA 


PERF 


Sum  of 
Squares 

df 

Mean  Square 

F 

Sig. 

Between  Groups 

2513,952 

1 

2513,952 

10,228 

,003 

Within  Groups 

7865,133 

32 

245,785 

Total 

10379,085 

33 

Table  N°  10.  ANOVA  results  for  Performance  variable,  H-B 


Since  Sig  =  0.003  is  much  lower  than  a  =  0.05,  the  Null  Hypothesis  “Ho”  must  be 
rejected,  which  means  that  there  exists  a  difference  between  the  means  of  both 
distributions  at  this  significance  level. 


We  state  that  the  performance  of  Group  4  HIERARCHY  USING  CIP  is 
significantly  superior  to  the  performance  of  Group  3  HIERARCHY  NO  CIP, 
within  a  confidence  interval  of  95%,  which  is  legitimately  attributable  to  the 
use  of  CIP. 

The  following  is  a  result  summary  for  Performance  variable: 


PERFORMANCE 

HIERARCHY 
NO  CIP 

HIERARCHY 
USING  CIP 

Hypothesis  Test 

Confidence 

Interval 

Mean 

22.35 

39.55 

There  exists 
significant 
difference 

95% 

Standard  Deviation 

17.98 

12.96 

Comparison  variable:  CIP  Usage 

Table  N°  11.  Results  summary  for  Performance  variable,  H-B. 


e.  ANOVA  results  for  mean  comparison  Share,  Post  and  Pull  variables,  H-B. 

We  practiced  an  ANOVA  procedure  to  compare  the  means  of  the  variables  Share, 
Post  and  Pull  of  Groups  3  and  4;  however  no  significant  differences  were  detected. 
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C.  TESTING  OF  HYPOTHESIS  “C”  (H-C) 

1.  EDGE  NO  CIP  -  HIERARCHY  NO  CIP 


Dependent  Variable 
Independent  Variable: 
Group  1: 

Group  3: 


P  =  Group  Performance 

Organizational  configuration  (Edge  v/s  Hierarchy) 
EDGE  NO  CIP 
HIERARCHY  NO  CIP 


a.  Performance  variable  distribution. 
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Figure  N°  9.  Boxplot  of  Performance  variable  Hypothesis  C. 

The  boxplot  above  shows  a  graphical  representation  of  the  distribution  data  of  both 
groups.  It  is  evident  that  the  “Edge  NO  CIP”  group  (Group  1)  outperformed  the 
“Hierarchy  No  CIP”  group  (Group  3). 

From  the  boxplot,  it  can  also  be  observed  that  the  overall  data  distribution  of  Group 
1  in  general  spans  through  higher  values  than  the  distribution  data  of  Group  3. 
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The  most  relevant  parameters  of  the  distribution  data  are  the  following: 


Performance 


N 

Mean 

Std.  Deviation 

Std.  Error 

95% Confidence  Interval  for 

Mean 

Minimum 

Maximum 

Lower  Bound 

Upper  Bound 

EdgeNOOP 

17 

38,4188 

27,28475 

6,61752 

24,3903 

52,4473 

,00 

111,36 

HenarchyNOOP 

17 

22,3524 

17,98712 

4,36252 

13,1042 

31,6005 

,00 

63,09 

Total 

34 

30,3856 

24,17238 

4,14553 

21,9514 

38,8197 

,00 

111,36 

Table  N°  12.  Relevant  parameters  of  Performance  distribution,  H-C 


b.  Compliance  of  conditions  for  ANOVA  hypothesis  test,  Performance  variable, 
H-C. 

We  practiced  the  Kolmogorov-Smimov  and  the  Shapiro-Wilk  tests  for  normality 
check.  We  also  verified  homogeneity  of  variances  through  the  Levene  test.  For  all 
tests  we  used  95%  of  confidence  (a  =  0.05).  Our  results  turned  out  satisfactory  as 
indicated  in  the  following  tables. 


Tests  of  Normality 


GROUP 

Kolmogorov-Smirnov  a 

Shapiro-Wilk  | 

Statistic 

df 

Sig. 

Statistic 

df 

Sig. 

Perf.  N°  1 

17 

,200* 

,927 

N°  3 

17 

,200* 

,935 

*.  This  is  a  lower  bound  of  the  true  significance, 
a.  Lilliefors  Significance  Correction 


Table  N°  13.  Test  of  normality  results,  H-C. 


Test  of  Homogeneity  of  Variances 

PERF 


Levene 

Statistic 

dfl 

df2 

Sig. 

1,949 

1 

32 

,172 

Table  N°  14.  Test  of  homogeneity  of  variance  results,  H-C 

The  value  of  the  parameter  Sig  =  0.172  is  greater  than  a  =  0.05.  This  verifies  the 
homogeneity  of  variance  of  the  distribution. 
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c.  ANOVA  results  for  mean  comparison  of  Performance  variable,  H-C. 

In  this  section  we  check  whether  the  mean  values  of  Performance  variable  of  both 
groups  differ  significantly  as  a  consequence  of  having  different  organizational 
structures. 

Null  Hypothesis  “Ho”:  The  means  of  the  distribution  of  Performance  variable  in 
both  groups  are  equal  (similar). 

ANOVA  results: 


ANOVA 


PERF 


Sum  of 
Squares 

df 

Mean  Square 

F 

Sig. 

Between  Groups 

2194,118 

1 

2194,118 

4,109 

,051 

Within  Groups 

17087,907 

32 

533,997 

Total 

19282,024 

33 

Table  N°  15.  ANOVA  results  for  Performance  variable,  H-C 


Since  Sig  =  0.051  is  higher  than  a  =  0.05,  the  Null  Hypothesis  “Ho”  cannot  be 
rejected,  which  means  that  there  is  no  significant  difference  between  the  means  of 
both  distributions  at  this  significance  level. 

We  state  that  the  performance  of  Group  1  EDGE  NO  CIP  is  not  significantly 
superior  to  the  performance  of  Group  3  HIERARCHY  NO  CIP,  within  a 
confidence  interval  of  95%. 


The  following  is  a  result  summary  for  Performance  variable: 


Performance 

EDGE 
NO  CIP 

HIERARCHY 
NO  CIP 

Hypothesis  Test 

Confidence 

Interval 

Mean 

38.42 

22.35 

There  exists  NO 
significant  difference 

95% 

Standard 

Deviation 

27.28 

17.98 

Comparison  variable:  Organizational  configuration 

Table  N°  16.  Results  summary  for  Performance  variable,  H-C. 


f.  ANOVA  results  for  mean  comparison  Share,  Post  and  Pull  variables,  H-C. 

We  practiced  an  ANOVA  procedure  to  compare  the  means  of  the  variables  Share, 
Post  and  Pull  of  Groups  1  and  3;  however  no  significant  differences  were  detected. 
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D.  TESTING  OF  HYPOTHESIS  “D”  (H-D) 

1.  COMPARING:  EDGE  USING  CIP  -  HIERARCHY  USING  CIP 

Dependent  Variable  :  P  =  Group  Performance 

Independent  Variable:  Organizational  configuration  (Edge  v/s  Hierarchy) 
Group  2 :  EDGE  USING  CIP 

Group  4:  HIERARCHY  USING  CIP 

a.  Performance  variable  distribution. 


EDGE  USING  CIP  HIERARCHY  USING  CIP 

GROUP 


Figure  N°  10.  Boxplot  of  Performance  variable  Hypothesis  D. 

The  boxplot  above  shows  a  graphical  representation  of  the  distribution  data  of  both 
groups.  It  is  evident  that  the  “Edge  USING  CIP”  group  (Group  2)  clearly 
outperformed  the  “Hierarchy  USING  CIP”  group  (Group  4). 

From  the  boxplot,  it  can  also  be  observed  that  the  central  quartiles  of  Group  2  span 
through  higher  values  than  those  of  Group  4,  even  though  the  dispersion  of  the 
former  data  set  is  greater. 
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The  most  relevant  parameters  of  the  distribution  data  are  the  following: 


Fferformance 


N 

IVfean 

Std.  Deviation 

Std.  Error 

95%  Confidence  Interval  for 

IVfean 

Mnirrum 

IVbxirrum 

Lower  Bound 

Uleper  Bound 

Edge  USING  OP 

17 

54,6935 

16,66355 

4,04150 

46,1259 

63,2611 

12,68 

74,35 

Herarchy  USING  OP 

17 

39,5500 

12,96281 

3,14394 

32,8851 

46,2149 

57,60 

Total 

34 

47,1218 

16,58825 

2,84486 

41,3339 

52,9097 

74,35 

Table  N°  17.  Relevant  parameters  of  Performance  distribution,  H-D 


b.  Compliance  of  conditions  for  ANOVA  hypothesis  test,  Performance  variable, 
H-D. 

We  practiced  the  Kolmogorov-Smimov  and  the  Shapiro-Wilk  tests  for  normality 
check.  We  also  verified  homogeneity  of  variances  through  the  Levene  test.  For  all 
tests  we  used  95%  of  confidence  (a  =  0.05).  Our  results  turned  out  satisfactory  as 
indicated  in  the  following  tables. 


Tests  of  Normality 


GROUP 

Kolmogorov-Smirnov  a 

Shapiro-Wilk 

Statistic 

df 

Sig. 

Statistic 

df 

Sig. 

Perf.  N°  2 

17 

,200* 

,919 

N°  4 _ 

17 

,047 

,851 

*.  This  is  a  lower  bound  of  the  true  significance, 
a.  Lilliefors  Significance  Correction 


Table  N°  18.  Test  of  normality  results,  H-D. 


Test  of  Homogeneity  of  Variances 


PERF 


Levene 

Statistic 

dfl 

df2 

Sig. 

1 

32 

,239 

Table  N°  19.  Test  of  homogeneity  of  variance  results,  H-D 


The  value  of  the  parameter  Sig  =  0.239  is  greater  than  a  =  0.05.  This  verifies  the 
homogeneity  of  variance  of  the  distribution. 
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c.  ANOVA  results  for  mean  comparison  of  Performance  variable,  H-D. 

In  this  section  we  check  whether  the  mean  values  of  Performance  variable  of  both 
groups  differ  significantly  as  a  consequence  of  having  different  organizational 
structures. 

Null  Hypothesis  “Ho”:  The  means  of  the  distribution  of  Performance  variable  in 
both  groups  are  equal  (similar). 

ANOVA  results: 

ANOVA 


PERF 


Sum  of 
Squares 

df 

Mean  Square 

F 

Sig. 

Between  Groups 

1949,275 

1 

1949,275 

8,747 

,006 

Within  Groups 

7131,332 

32 

222,854 

Total 

9080,607 

33 

Table  N°  20.  ANOVA  results  for  Performance  variable,  H-D 


Since  Sig  =  0.006  is  much  lower  than  a  =  0.05,  the  Null  Hypothesis  “Ho”  must  be 
rejected,  which  means  that  there  exists  a  difference  between  the  means  of  both 
distributions  at  this  significance  level. 

We  state  that  the  performance  of  Group  2  EDGE  USING  CIP  is  significantly 
superior  to  the  performance  of  Group  4  HIERARCHY  USING  CIP,  within  a 
confidence  interval  of  95%,  which  is  legitimately  attributable  to  the  difference 
in  organizational  structure  between  both  groups. 


The  following  is  a  result  summary  for  Performance  variable: 


Performance 

EDGE 
USING  CIP 

HIERARCHY 
USING  CIP 

Hypothesis  Test 

Confidence 

Interval 

Mean 

54.69 

39.55 

There  exists 
significant 
difference 

95% 

Standard  Deviation 

16.66 

12.96 

Comparison  variable:  Organizational  configuration 

Table  N°  21.  Results  summary  for  Performance  variable,  H-D. 


g.  ANOVA  results  for  mean  comparison  Share,  Post  and  Pull  variables,  H-D. 

We  practiced  an  ANOVA  procedure  to  compare  the  means  of  the  variables  Share, 
Post  and  Pull  of  Groups  2  and  4;  however  no  significant  differences  were  detected. 
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E.  TESTING  OF  HYPOTHESIS  “E”  (H-E) 

1.  COMPARING:  EDGE  NO  CIP  -  HIERARCHY  USING  CIP 


Dependent  Variable  : 
Independent  Variable: 

Group  1: 

Group  4: 


P  =  Group  Performance 

Organizational  configuration  (Edge  v/s  Hierarchy);  CIP 
usage. 

EDGE  NO  CIP 
HIERARCHY  USING  CIP 


d.  Performance  variable  distribution. 
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The  boxplot  above  shows  a  graphical  representation  of  the  distribution  data  of  both 
groups.  It  is  evident  that  the  “Edge  NO  CIP”  group  (Group  1)  exhibits  A 
performance  that  is  similar  to  that  of  the  “Hierarchy  USING  CIP”  group  (Group  4). 


- 1 - 1 - 

EDGE  NO  CIP  HIERARCHY  USING  CIP 

GROUP  1 

Figure  N°  11.  Boxplot  of  Performance  variable. 
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From  the  boxplot,  it  can  also  be  observed  that  the  central  quartiles  of  Group  1  span 
through  similar  values  than  those  of  Group  4,  however  the  data  dispersion  of  the 
former  group  is  greater. 


The  most  relevant  parameters  of  the  distribution  data  are  the  following: 

Descriptives 


Perform 


N 

Mean 

Std.  Deviation 

Std.  Error 

95%  Confidence  Interval  for 
Mean 

Minimum 

Maximum 

Lower  Bound 

Upper  Bound 

Edge  NO  CIP 

17 

38,4188 

27,28475 

6,61752 

24,3903 

52,4473 

,00 

111,36 

Hierar  USING  CIP 

17 

39,5500 

12,96281 

3,14394 

32,8851 

46,2149 

,00 

57,60 

Total 

34 

38,9844 

21,04163 

3,60861 

31,6426 

46,3262 

,00 

111,36 

Table  N°  22.  Relevant  parameters  of  Performance  distribution,  H-E. 


e.  Compliance  of  conditions  for  ANOVA  hypothesis  test,  Performance  variable. 
H-E. 

We  practiced  the  Kolmogorov-Smimov  and  the  Shapiro-Wilk  tests  for  normality 
check.  We  also  verified  homogeneity  of  variances  through  the  Levene  test.  For  all 
tests  we  used  95%  of  confidence  (a  =  0.05). 


Tests  of  Normality 


Kolmogorov-Smirnov(a) 

Shapiro-Wilk 

Group 

Statistic 

df 

Sig. 

Statistic 

df 

Sig. 

„  ^  Edge  NO  CIP 

Perform 

,132 

17 

,200(*) 

,927 

17 

,191 

Hierar  USING  CIP 

,209 

17 

,047 

,851 

17 

,011 

*  This  is  a  lower  bound  of  the  true  significance, 
a  Lilliefors  Significance  Correction 


Table  N°  23.  Test  of  normality  results,  H-E. 


Test  of  Homogeneity  of  Variances 


PERF 


Levene 

Statistic 

dfl 

df2 

Sig. 

7,106 

1 

32 

,012 

Table  N°  24.  Test  of  homogeneity  of  variance  results,  H-E. 


-38- 


Testing  Edge  versus  Hierarchy  C2  Organizations  using 
the  ELICIT  platform  and  Common  Identification  Picture  tool 


The  value  of  the  parameter  Sig  =  0.012  is  lower  than  a  =  0.05.  This  indicates  that 
there  is  no  homogeneity  of  variance  of  the  distribution.  Therefore  it  is  not  possible 
to  use  ANOVA  to  compare  the  performance  of  these  two  groups.  Alternatively,  we 
applied  the  “non  parametric”  Kruskal- Wallis  Test  for  mean  comparison,  as 
recommended  in  Camacho,  2006. 

f.  Kruskal-Wallis  Test  results  for  mean  comparison  of  Performance  variable,  H- 
E. 

In  this  section  we  check  whether  the  mean  values  of  Performance  variable  of  both 
groups  differ  significantly  as  a  consequence  of  having  different  organizational 
structures  and  using  different  interaction  tools. 

Null  Hypothesis  “Ho”:  The  means  of  the  distribution  of  Performance  variable  in 
both  groups  are  equal  (similar). 

Kruscal-Wallis  Test  results: 

Test  Statisticsa,b 


Perform 

Chi-Square 

,428 

df 

1 

Asymp.  Sig. 

,513 

a.  Kruskal  Wallis  Test 

b.  Grouping  Variable:  Group 

Table  N°  25.  Kruscal-Wallis  Test  results  for  Performance  variable,  H-E. 

Since  Sig  =  0.513  is  much  higher  than  a  =  0.05,  the  Null  Hypothesis  “Ho”  cannot 
be  rejected,  which  means  that  there  exists  no  difference  between  the  means  of  both 
distributions  at  this  significance  level. 

We  state  that  the  performance  of  Group  1  EDGE  NO  CIP  is  similar  to  the 
performance  of  Group  4  HIERARCHY  USING  CIP,  within  a  confidence 
interval  of  95%,  which  is  legitimately  attributable  to  the  combined  differences 
in  organizational  structure  between  both  groups  and  the  usage  of  the  CIP  tool. 
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The  following  is  a  result  summary  for  Performance  variable: 


Performance 

EDGE 

NO  CIP 

HIERARCHY 
USING  CIP 

Hypothesis  Test 

Confidence 

Interval 

Mean 

54.69 

39.55 

There  exists  no 
significant 
difference 

95% 

Standard  Deviation 

16.66 

12.96 

Comparison  variable:  Organizational  configuration 

Table  N°  26.  Results  summary  for  Performance  variable,  H-D. 


h.  ANOVA  results  for  mean  comparison  Share,  Post  and  Pull  variables,  H-E. 

We  practiced  an  ANOVA  procedure  to  compare  the  means  of  the  variables  Share, 
Post  and  Pull  of  Groups  1  and  4;  however  no  significant  differences  were  detected. 
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To  present  experimental  research  conducted  by  the  Chilean  Army,  which 
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For  the  latter,  we  combined  ELICIT  with  a  complementary  tool  we  call 
Common  Identification  Picture  (CIP),  which  enables  participants  to  share 
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ELICIT:  A  PROBLEM  SOLVING  SIMULATION  GAME 


17  individuals  organized  either  as  a  Hierarchical  or  as  an  Edge  structure. 

Players  collaborate  within  a  networked  environment  by  sharing  information  between 
each  other  or  by  posting  and  pulling  from  websites. 

The  purpose  is  to  complete  the  threat  recognition  by  identifying  Who,  What,  Where  and 
When  the  attack  will  occur. 
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ELICIT:  A  PROBLEM  SOLVING  SIMULATION  GAME 


•  ELICIT  provides  the  players  with  simple  pieces  of  information  (factoids)  of 
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share  them  to  improve  the  collective  awareness  of  the  situation. 

•  Performance  is  measured  in  terms  of  accuracy  and  timeliness  of  threat 
identification. 
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•  Restricted  Information 


Hierarchical  Organization,  “Who”  group  participant,  8  IDs  already  made 
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COMMON  IDENTIFICATION  PICTURE  TOOL  (CIP) 

CIP  USER  INTERFACE 


m  -  c  X  ^  CD.  http :://!  o  c  a  I  h  o  st:S080/CIP/g  raf2  .j  s  p 


'Cf  T  Google 


|  Cornenzara  usar  Firefox  S  Ultimas  noticias  |  |  Q  AdminGF  Q  LOGIN  CIP  Q  Inicio  CIP  Qj  Multiseries  Banco  en  Lmea  -  Santa,.,  El  Comms  yj)  Java  Sites 

Common  Information  Picture  [CIP)  + 


COMMON  IDENTIFICATION  PICTURE 


Who 


What 


Where 


User:  whitley 
Area:  Transversal 

Experiment  Date:  2011  01  25 19:17:47 
Enter  your  attack  identifi  cation 


Who... 

What... 

Where... 

when...  Day  ▼  Month  ▼  Hour 


Send  Clear 


My  Identifications 

{NE  Hour  Who  What  Where  When) 


HYPOTHESES 


Dependent  Variable:  Group  Performance. 

Independent  Variables:  CIP  Usage,  Organizational  Configuration. 


Whenever  structure  is  kept  constant,  CIP  usage  will  result  in  better  performance. 

Edge  organization  will  outperform  Hierarchies,  both  using  and  without  using  CIP. 

The  performance  difference  between  a  Hierarchical  organization  using  CIP  and  an  Edge 
organization  without  using  CIP  will  be  meager. 


DATA  ANALYSIS 
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GROUP  PERFORMANCE  COMPARISON 


Performance:  P  =  K*  (A/T) ;  where:  K  is  100,000  (constant  for  figure  readability) 

A  is  Accuracy  (correctness  of  IDs) 

T  is  Time  (time  to  submit  IDs) 


Data  Analysis  was  made  in  two  steps: 

1.  Obtain  a  comparative  boxplot  in  order  to  examine  and  contrast  the  main  parameters  of 
the  data  distributions. 

2.  Run  an  ANOVA  hypothesis  test  using  95%  of  confidence  (a  =  0.05): 

•  Check  for  normality  through  Kolmogorov-Smirnov  and  Shapiro-Wilk  tests. 

•  Verify  homogeneity  of  variances  through  the  Levene  test. 

Perform  a  mean  comparison  through  ANOVA  and  -whenever  possible-  rejected  the 
corresponding  null  hypothesis. 


If  ANOVA  was  not  applicable,  we  applied  the  non  parametric  Kruskal-Wallis  Test  for  mean 
comparison. 
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RESULTS 


ID 

HIPOTHESES 

P 

(a  =  0.05) 

RESULT 

KEY 

FACTOR * 

A 

Group  2  “ Edge  Using  CIP”  outperforms 
Group  1  “Edge  No  CIP” 

0.044 

Validated 

Accuracy 

B 

A 

Group  4  “Hierarchical  Using  CIP” 
outperforms  Group  3  “Hierarchical  No  CIP” 

0.003 

Validated 

Accuracy 

c 

Group  1  “Edge  No  CIP”  outperforms  Group 

3  “Hierarchical  No  CIP” 

0.051 

Rejected 

None 

D 

Group  2  “Edge  Using  CIP”  outperforms 
Group  4  “Hierarchical  Using  CIP” 

0.006 

Validated 

Accuracy, 

Time 

E 

Group  1  “Edge  No  CIP”  and  Group  4 
“Hierarchical  Using  CIP”  perform  similarly. 

0.513 

Validated 

Accuracy, 

Time 

*  KEY  FACTOR:  is  the  factor  that  influences  the  most  in  the  result  of  Performance  variable  “P”,  which  is  computed 
through  the  equation:  P  =  K*  (A/T) 


RESULT  DISCUSION 


•  ELICIT+CIP  improved  performance  of  both  types  of  organizations. 

-  CIP  added  realistic  conditions  to  the  experimental  setting. 

-  CIP  enhances  abilities  in  two  of  the  C2  maturity  dimensions:  Patterns  of  Interactions  and 
Distribution  of  Information . 

•  CIP  moderately  slows  down  decision  making.  However,  it  considerably  increases 
accuracy,  as  it  provides  access  to  notions  that  have  the  potential  to  influence  action. 

•  Performance  difference  between  Edge  and  Hierarchies  was  scarce  without  CIP,  but 
discrepancy  was  significant  when  using  CIP. 

-  Incremented  interactions  and  enhanced  cognitive  teamwork  benefits  more  the  less  constrained 
organization. 

-  Enrichment  of  interaction  space  is  better  exploited  by  the  entity  featuring  a  more  complete 
pattern  of  interactions  and  freedom  to  collaborate. 

-  The  organization  type  that  is  better  able  to  exploit  the  new  functionalities  reaped  more  benefits 
out  of  CIP. 

•  By  improving  interaction  means  and  raising  the  cognitive  level  of  collaboration,  Hierarchies 
can  mitigate  their  communicational  limitations,  matching  the  performance  of  Edge  entities. 


CONCLUSIONS 


The  CIP  tool  allows  the  players  to  share  their  understanding  of  the  situation  after 
analyzing  the  pieces  of  information  provided  by  ELICIT. 

When  units  make  efforts  to  communicate  their  understanding  of  the  situation,  it 
becomes  easier  for  the  collective  to  form  “correct”  shared  awareness. 

The  practice  of  performing  information  analyses,  and  diffusing  intelligence  at  all  levels 
should  be  stressed  and  deeply  embedded  in  doctrinal  practices. 

Less  restricted  organizations  are  better  able  to  exploit  knowledge  sharing 
functionalities.  Only  when  these  features  are  implemented,  the  noticeable  difference 
emerges. 

This  research  supports  “NNC2MM  “  theory  as  Edge  organizations  do  outperform 
Hierarchies.  It  is  correct  to  adopt  Edge  whenever  the  situation  allows  it. 

To  materialize  Edge  superior  performance,  all  the  potential  of  NCW  must  be  available, 
specifically,  rich  communication  channels;  information  diffusion,  knowledge  sharing, 
among  others. 

the  architectural  communication  restrictions  of  Hierarchies  can  be  reduced  by 
implementing  technology  and  procedures  that  encourage  intelligence  sharing. 


